adult

Dataset statistics

Number of variables	12
Number of observations	48842
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	4.5 MiB
Average record size in memory	96.0 B

Variable types

NUM	8
CAT	3
BOOL	1

Reproduction

Analysis started	2020-06-24 15:10:00.166747
Analysis finished	2020-06-24 15:10:15.231039
Duration	15.06 seconds
Version	pandas-profiling v2.8.0
Command line	`pandas_profiling --config_file config.yaml [YOUR_FILE.csv]`
Download configuration	config.yaml

Warnings

`Unnamed: 0` has unique values	Unique
`score` has 13093 (26.8%) zeros	Zeros
`capital-gain` has 44807 (91.7%) zeros	Zeros
`capital-loss` has 46560 (95.3%) zeros	Zeros

Unnamed: 0
Real number (ℝ_≥0)

UNIQUE

Distinct count	48842
Unique (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	24420.5
Minimum	0
Maximum	48841
Zeros	1
Zeros (%)	< 0.1%
Memory size	381.6 KiB

Quantile statistics

Minimum	0
5-th percentile	2442.05
Q1	12210.25
median	24420.5
Q3	36630.75
95-th percentile	46398.95
Maximum	48841
Range	48841
Interquartile range (IQR)	24420.5

Descriptive statistics

Standard deviation	14099.61526
Coefficient of variation (CV)	0.5773680007
Kurtosis	-1.2
Mean	24420.5
Median Absolute Deviation (MAD)	12210.5
Skewness	0
Sum	1192746061
Variance	198799150.5

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
2047	1	< 0.1%
7497	1	< 0.1%
34138	1	< 0.1%
40281	1	< 0.1%
38232	1	< 0.1%
11599	1	< 0.1%
9550	1	< 0.1%
15693	1	< 0.1%
13644	1	< 0.1%
3403	1	< 0.1%
Other values (48832)	48832	> 99.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1	< 0.1%
1	1	< 0.1%
2	1	< 0.1%
3	1	< 0.1%
4	1	< 0.1%

Value	Count	Frequency (%)
48841	1	< 0.1%
48840	1	< 0.1%
48839	1	< 0.1%
48838	1	< 0.1%
48837	1	< 0.1%

income
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	381.6 KiB

0	37155
1	11687

Frequency Table

Value	Count	Frequency (%)
0	37155	76.1%
1	11687	23.9%

score
Real number (ℝ_≥0)

ZEROS

Distinct count	437
Unique (%)	0.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.242394501902149
Minimum	0.0
Maximum	1.0
Zeros	13093
Zeros (%)	26.8%
Memory size	381.6 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0.06
Q3	0.37
95-th percentile	0.98
Maximum	1
Range	1
Interquartile range (IQR)	0.37

Descriptive statistics

Standard deviation	0.3289513128
Coefficient of variation (CV)	1.357090653
Kurtosis	-0.06091413434
Mean	0.2423945019
Median Absolute Deviation (MAD)	0.06
Skewness	1.216295538
Sum	11839.03226
Variance	0.1082089662

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	13093	26.8%
0.01	3522	7.2%
0.02	2218	4.5%
0.03	1764	3.6%
1	1722	3.5%
0.04	1488	3.0%
0.05	1221	2.5%
0.06	1064	2.2%
0.08	962	2.0%
0.07	923	1.9%
Other values (427)	20865	42.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	13093	26.8%
0.0025	5	< 0.1%
0.003333333333	4	< 0.1%
0.004	2	< 0.1%
0.005	7	< 0.1%

Value	Count	Frequency (%)
1	1722	3.5%
0.99	465	1.0%
0.98	262	0.5%
0.97	225	0.5%
0.96	226	0.5%

gender
Categorical

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	381.6 KiB

Male	32650
Female	16192

Common Values
Length

Value	Count	Frequency (%)
Male	32650	66.8%
Female	16192	33.2%

Length

Max length	6
Median length	4
Mean length	4.663035912
Min length	4

race
Categorical

Distinct count	5
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	381.6 KiB

White	41762
Black	4685
Asian-Pac-Islander	1519
Amer-Indian-Eskimo	470
Other	406

Common Values
Length

Value	Count	Frequency (%)
White	41762	85.5%
Black	4685	9.6%
Asian-Pac-Islander	1519	3.1%
Amer-Indian-Eskimo	470	1.0%
Other	406	0.8%

Length

Max length	18
Median length	5
Mean length	5.529400925
Min length	5

marital-status
Categorical

Distinct count	7
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	381.6 KiB

Married-civ-spouse	22379
Never-married	16117
Divorced	6633
Separated	1530
Widowed	1518
Other values (2)	665

Common Values
Length

Value	Count	Frequency (%)
Married-civ-spouse	22379	45.8%
Never-married	16117	33.0%
Divorced	6633	13.6%
Separated	1530	3.1%
Widowed	1518	3.1%
Married-spouse-absent	628	1.3%
Married-AF-spouse	37	0.1%

Length

Max length	21
Median length	13
Mean length	14.40604398
Min length	7

age
Real number (ℝ_≥0)

Distinct count	74
Unique (%)	0.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	38.64358543876172
Minimum	17
Maximum	90
Zeros	0
Zeros (%)	0.0%
Memory size	381.6 KiB

Quantile statistics

Minimum	17
5-th percentile	19
Q1	28
median	37
Q3	48
95-th percentile	63
Maximum	90
Range	73
Interquartile range (IQR)	20

Descriptive statistics

Standard deviation	13.71050993
Coefficient of variation (CV)	0.35479394
Kurtosis	-0.1842687406
Mean	38.64358544
Median Absolute Deviation (MAD)	10
Skewness	0.5575803166
Sum	1887430
Variance	187.9780827

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
36	1348	2.8%
35	1337	2.7%
33	1335	2.7%
23	1329	2.7%
31	1325	2.7%
34	1303	2.7%
37	1280	2.6%
28	1280	2.6%
30	1278	2.6%
38	1264	2.6%
Other values (64)	35763	73.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
17	595	1.2%
18	862	1.8%
19	1053	2.2%
20	1113	2.3%
21	1096	2.2%

Value	Count	Frequency (%)
90	55	0.1%
89	2	< 0.1%
88	6	< 0.1%
87	3	< 0.1%
86	1	< 0.1%

fnlwgt
Real number (ℝ_≥0)

Distinct count	28523
Unique (%)	58.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	189664.13459727284
Minimum	12285
Maximum	1490400
Zeros	0
Zeros (%)	0.0%
Memory size	381.6 KiB

Quantile statistics

Minimum	12285
5-th percentile	39615.4
Q1	117550.5
median	178144.5
Q3	237642
95-th percentile	379481.65
Maximum	1490400
Range	1478115
Interquartile range (IQR)	120091.5

Descriptive statistics

Standard deviation	105604.0254
Coefficient of variation (CV)	0.5567949135
Kurtosis	6.057848212
Mean	189664.1346
Median Absolute Deviation (MAD)	60295.5
Skewness	1.438891879
Sum	9263575662
Variance	1.115221019e+10

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
203488	21	< 0.1%
190290	19	< 0.1%
120277	19	< 0.1%
125892	18	< 0.1%
126569	18	< 0.1%
126675	17	< 0.1%
113364	17	< 0.1%
99185	17	< 0.1%
186934	16	< 0.1%
111567	16	< 0.1%
Other values (28513)	48664	99.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
12285	1	< 0.1%
13492	1	< 0.1%
13769	3	< 0.1%
13862	1	< 0.1%
14878	1	< 0.1%

Value	Count	Frequency (%)
1490400	1	< 0.1%
1484705	1	< 0.1%
1455435	1	< 0.1%
1366120	1	< 0.1%
1268339	1	< 0.1%

education-num
Real number (ℝ_≥0)

Distinct count	16
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	10.078088530363212
Minimum	1
Maximum	16
Zeros	0
Zeros (%)	0.0%
Memory size	381.6 KiB

Quantile statistics

Minimum	1
5-th percentile	5
Q1	9
median	10
Q3	12
95-th percentile	14
Maximum	16
Range	15
Interquartile range (IQR)	3

Descriptive statistics

Standard deviation	2.570972756
Coefficient of variation (CV)	0.2551051966
Kurtosis	0.6257452728
Mean	10.07808853
Median Absolute Deviation (MAD)	1
Skewness	-0.3165248567
Sum	492234
Variance	6.60990091

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
9	15784	32.3%
10	10878	22.3%
13	8025	16.4%
14	2657	5.4%
11	2061	4.2%
7	1812	3.7%
12	1601	3.3%
6	1389	2.8%
4	955	2.0%
15	834	1.7%
Other values (6)	2846	5.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	83	0.2%
2	247	0.5%
3	509	1.0%
4	955	2.0%
5	756	1.5%

Value	Count	Frequency (%)
16	594	1.2%
15	834	1.7%
14	2657	5.4%
13	8025	16.4%
12	1601	3.3%

capital-gain
Real number (ℝ_≥0)

ZEROS

Distinct count	123
Unique (%)	0.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	1079.0676262233324
Minimum	0
Maximum	99999
Zeros	44807
Zeros (%)	91.7%
Memory size	381.6 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	0
95-th percentile	5013
Maximum	99999
Range	99999
Interquartile range (IQR)	0

Descriptive statistics

Standard deviation	7452.019058
Coefficient of variation (CV)	6.905979641
Kurtosis	152.6930963
Mean	1079.067626
Median Absolute Deviation (MAD)	0
Skewness	11.894659
Sum	52703821
Variance	55532588.04

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	44807	91.7%
15024	513	1.1%
7688	410	0.8%
7298	364	0.7%
99999	244	0.5%
3103	152	0.3%
5178	146	0.3%
5013	117	0.2%
4386	108	0.2%
8614	82	0.2%
Other values (113)	1899	3.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	44807	91.7%
114	8	< 0.1%
401	5	< 0.1%
594	52	0.1%
914	10	< 0.1%

Value	Count	Frequency (%)
99999	244	0.5%
41310	3	< 0.1%
34095	6	< 0.1%
27828	58	0.1%
25236	14	< 0.1%

capital-loss
Real number (ℝ_≥0)

ZEROS

Distinct count	99
Unique (%)	0.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	87.50231358257237
Minimum	0
Maximum	4356
Zeros	46560
Zeros (%)	95.3%
Memory size	381.6 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	0
95-th percentile	0
Maximum	4356
Range	4356
Interquartile range (IQR)	0

Descriptive statistics

Standard deviation	403.0045521
Coefficient of variation (CV)	4.605644532
Kurtosis	20.01434595
Mean	87.50231358
Median Absolute Deviation (MAD)	0
Skewness	4.569808858
Sum	4273788
Variance	162412.669

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	46560	95.3%
1902	304	0.6%
1977	253	0.5%
1887	233	0.5%
2415	72	0.1%
1485	71	0.1%
1848	67	0.1%
1590	62	0.1%
1602	62	0.1%
1876	59	0.1%
Other values (89)	1099	2.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	46560	95.3%
155	1	< 0.1%
213	5	< 0.1%
323	5	< 0.1%
419	3	< 0.1%

Value	Count	Frequency (%)
4356	3	< 0.1%
3900	2	< 0.1%
3770	4	< 0.1%
3683	2	< 0.1%
3175	2	< 0.1%

hours-per-week
Real number (ℝ_≥0)

Distinct count	96
Unique (%)	0.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	40.422382375824085
Minimum	1
Maximum	99
Zeros	0
Zeros (%)	0.0%
Memory size	381.6 KiB

Quantile statistics

Minimum	1
5-th percentile	17.05
Q1	40
median	40
Q3	45
95-th percentile	60
Maximum	99
Range	98
Interquartile range (IQR)	5

Descriptive statistics

Standard deviation	12.39144402
Coefficient of variation (CV)	0.3065490774
Kurtosis	2.95105909
Mean	40.42238238
Median Absolute Deviation (MAD)	3
Skewness	0.2387496572
Sum	1974310
Variance	153.547885

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
40	22803	46.7%
50	4246	8.7%
45	2717	5.6%
60	2177	4.5%
35	1937	4.0%
20	1862	3.8%
30	1700	3.5%
55	1051	2.2%
25	958	2.0%
48	770	1.6%
Other values (86)	8621	17.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	27	0.1%
2	53	0.1%
3	59	0.1%
4	84	0.2%
5	95	0.2%

Value	Count	Frequency (%)
99	137	0.3%
98	14	< 0.1%
97	2	< 0.1%
96	9	< 0.1%
95	2	< 0.1%

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

Count
Matrix

First rows

	Unnamed: 0	income	score	gender	race	marital-status	age	fnlwgt	education-num	capital-gain	hours-per-week
0	0	0	0.00	Male	Black	Never-married	25	226802	7	0	40
1	1	0	0.22	Male	White	Married-civ-spouse	38	89814	9	0	50
2	2	1	0.95	Male	White	Married-civ-spouse	28	336951	12	0	40
3	3	1	1.00	Male	Black	Married-civ-spouse	44	160323	10	7688	40
4	4	0	0.00	Female	White	Never-married	18	103497	10	0	30
5	5	0	0.00	Male	White	Never-married	34	198693	6	0	30
6	6	0	0.00	Male	Black	Never-married	29	227026	9	0	40
7	7	1	0.44	Male	White	Married-civ-spouse	63	104626	15	3103	32
8	8	0	0.00	Female	White	Never-married	24	369667	10	0	40
9	9	0	0.01	Male	White	Married-civ-spouse	55	104996	4	0	10

Last rows

	Unnamed: 0	income	score	gender	race	marital-status	age	fnlwgt	education-num	capital-gain	hours-per-week
48832	48832	0	0.01	Male	Amer-Indian-Eskimo	Married-civ-spouse	32	34066	6	0	40
48833	48833	0	0.26	Male	White	Married-civ-spouse	43	84661	11	0	45
48834	48834	0	0.04	Male	Asian-Pac-Islander	Never-married	32	116138	14	0	11
48835	48835	1	0.76	Male	White	Married-civ-spouse	53	321865	14	0	40
48836	48836	0	0.00	Male	White	Never-married	22	310152	10	0	40
48837	48837	0	0.04	Female	White	Married-civ-spouse	27	257302	12	0	38
48838	48838	1	0.73	Male	White	Married-civ-spouse	40	154374	9	0	40
48839	48839	0	0.03	Female	White	Widowed	58	151910	9	0	40
48840	48840	0	0.00	Male	White	Never-married	22	201490	9	0	20
48841	48841	1	1.00	Female	White	Married-civ-spouse	52	287927	9	15024	40

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Cramér's V (φc)

Missing values

Sample

First rows

Last rows